Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

联合学习（FL），其中多个机构在不共享数据的情况下协作训练机器学习模型正在变得流行。参与机构可能不会平等地做出贡献，有些贡献了更多的数据，一些更好的质量数据或一些更多样化的数据。为了公平地排名不同机构的贡献，沙普利价值（SV）已成为选择方法。精确的SV计算非常昂贵，尤其是在有数百个贡献者的情况下。现有的SV计算技术使用近似值。但是，在医疗保健中，贡献机构的数量可能不是巨大的规模，计算精确的SVS仍然很昂贵，但并非不可能。对于此类设置，我们提出了一种称为Safe的高效SV计算技术（用于使用Enembly的联合学习的Shapley值）。我们从经验上表明，安全计算接近精确SV的值，并且其性能优于当前SV近似值。这在医学成像环境中尤其重要，在医学成像环境中，整个机构之间的广泛异质性猖ramp，并且需要快速准确的数据评估来确定每个参与者在多机构协作学习中的贡献。

乳腺癌是最常见的癌症，乳房X线摄影筛查的早期检测对于改善患者预后至关重要。评估乳房X线乳房密度在临床上很重要，因为浓密的乳房具有更高的风险，并且更有可能阻塞肿瘤。专家的手动评估既耗时又受评估者间的可变性。因此，对乳房X线乳房密度评估的深度学习方法的发展有所增加。尽管深度学习在乳房X线摄影的应用中表现出了令人印象深刻的表现，但在仍然相对较少的深度学习系统中的临床部署中；从历史上看，乳房X线摄影计算机辅助诊断（CAD）已过分宣传，无法提供。这部分是由于无法直观地量化临床医生算法的不确定性，这将大大提高可用性。共形预测非常适合增加对深度学习工具的可靠和信任，但它们缺乏对医疗数据集的现实评估。在本文中，我们介绍了应用于医学成像任务的三个可能应用的详细分析：分配转移表征，预测质量的改善和亚组公平分析。我们的结果表明，无分配不确定性量化技术的潜力可以增强对AI算法的信任并加快其翻译为使用。

深度学习（DL）模型为各种医学成像基准挑战提供了最先进的性能，包括脑肿瘤细分（BRATS）挑战。然而，局灶性病理多隔室分割（例如，肿瘤和病变子区）的任务特别具有挑战性，并且潜在的错误阻碍DL模型转化为临床工作流程。量化不确定形式的DL模型预测的可靠性，可以实现最不确定的地区的临床审查，从而建立信任并铺平临床翻译。最近，已经引入了许多不确定性估计方法，用于DL医学图像分割任务。开发指标评估和比较不确定性措施的表现将有助于最终用户制定更明智的决策。在本研究中，我们探索并评估在Brats 2019-2020任务期间开发的公制，以对不确定量化量化（Qu-Brats），并旨在评估和排列脑肿瘤多隔室分割的不确定性估计。该公制（1）奖励不确定性估计，对正确断言产生高置信度，以及在不正确的断言处分配低置信水平的估计数，（2）惩罚导致更高百分比的无关正确断言百分比的不确定性措施。我们进一步基准测试由14个独立参与的Qu-Brats 2020的分割不确定性，所有这些都参与了主要的Brats细分任务。总体而言，我们的研究结果证实了不确定性估计提供了分割算法的重要性和互补价值，因此突出了医学图像分析中不确定性量化的需求。我们的评估代码在HTTPS://github.com/ragmeh11/qu-brats公开提供。

尽管对临床机器学习研究有强烈的关注和相当大的投资，但在现实世界的临床环境中，在大规模的应用中已经部署了相对较少的应用。虽然研究在推进最先进的情况下很重要，但翻译同样重要的是，使这些技术和技术能够最终影响医疗保健。我们认为对几个考虑缺乏升值是在期望和现实之间这种差异的主要原因。为了更好地描述研究人员和从业者之间的整体视角，我们调查了几个从业人员在开发CML中进行临床部署的商业经验。使用这些洞察力，我们确定了几个主要类别的挑战，以便更好地设计和开发临床机学习应用。

The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in the conservation of wildlife. In this article, we will use case studies to demonstrate the importance of designing conservation tools with human-wildlife interaction in mind and provide a framework for creating successful tools. These case studies include a range of complexities, from simple cat collars to machine learning and game theory methodologies. Our goal is to introduce and inform current and future researchers in the field of conservation technology and provide references for educating the next generation of conservation technologists. Conservation technology not only has the potential to benefit biodiversity but also has broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.

We present Muse, a text-to-image Transformer model that achieves state-of-the-art image generation performance while being significantly more efficient than diffusion or autoregressive models. Muse is trained on a masked modeling task in discrete token space: given the text embedding extracted from a pre-trained large language model (LLM), Muse is trained to predict randomly masked image tokens. Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding. The use of a pre-trained LLM enables fine-grained language understanding, translating to high-fidelity image generation and the understanding of visual concepts such as objects, their spatial relationships, pose, cardinality etc. Our 900M parameter model achieves a new SOTA on CC3M, with an FID score of 6.06. The Muse 3B parameter model achieves an FID of 7.88 on zero-shot COCO evaluation, along with a CLIP score of 0.32. Muse also directly enables a number of image editing applications without the need to fine-tune or invert the model: inpainting, outpainting, and mask-free editing. More results are available at https://muse-model.github.io

An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-balanced Re-weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating the deteriorating performances of the minority predicate predictions, showing drastic dropping recall scores, i.e., losing the majority predicate performances. It has not yet correctly analyzed the trade-off between majority and minority predicate performances in the limited SGG datasets. In this paper, to alleviate the issue, the Skew Class-balanced Re-weighting (SCR) loss function is considered for the unbiased SGG models. Leveraged by the skewness of biased predicate predictions, the SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones. Extensive experiments conducted on the standard Visual Genome dataset and Open Image V4 \& V6 show the performances and generality of the SCR with the traditional SGG models.

With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples. It has been a new trend exploring ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress, challenges, and future work in ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques of ICL, including training strategies, prompting strategies, and so on. Finally, we present the challenges of ICL and provide potential directions for further research. We hope our work can encourage more research on uncovering how ICL works and improving ICL in future work.

Masked image modeling (MIM) has shown great promise for self-supervised learning (SSL) yet been criticized for learning inefficiency. We believe the insufficient utilization of training signals should be responsible. To alleviate this issue, we introduce a conceptually simple yet learning-efficient MIM training scheme, termed Disjoint Masking with Joint Distillation (DMJD). For disjoint masking (DM), we sequentially sample multiple masked views per image in a mini-batch with the disjoint regulation to raise the usage of tokens for reconstruction in each image while keeping the masking rate of each view. For joint distillation (JD), we adopt a dual branch architecture to respectively predict invisible (masked) and visible (unmasked) tokens with superior learning targets. Rooting in orthogonal perspectives for training efficiency improvement, DM and JD cooperatively accelerate the training convergence yet not sacrificing the model generalization ability. Concretely, DM can train ViT with half of the effective training epochs (3.7 times less time-consuming) to report competitive performance. With JD, our DMJD clearly improves the linear probing classification accuracy over ConvMAE by 5.8%. On fine-grained downstream tasks like semantic segmentation, object detection, etc., our DMJD also presents superior generalization compared with state-of-the-art SSL methods. The code and model will be made public at https://github.com/mx-mark/DMJD.